Traditionally, high quality motion picture images are captured by a photographic film. The primary benefits of film over other image capture technologies for motion picture applications include wide exposure dynamic range, preferred color reproduction control, a desired level of texture (image grain), fast photographic speed, high resolution and sharpness, and flexibility in framing through various post-production operations. In general, motion picture images are used for exhibition in either theatrical projection or broadcast television distribution.
Color images on photographic film are produced by exposing light-sensitive layers coated on a flexible semi-transparent support through a film camera and lens, and subjecting the film to prescribed chemical amplification processes to produce dyes with a predictable optical density profile. In typical motion picture applications, the original captured film image can be further printed optically onto another piece of intermediate or print motion picture film as suitable for theatrical projection or the optical density signature recorded on the film can be transferred to digital data on a film scanner. Once scanned, image manipulation can be imparted by way of a color corrector or various other digital intermediate techniques suitable for preparing the image for distribution (television broadcast, theatrical projection, etc.).
Photographic color negative films create complementary cyan, magenta, and yellow dye amounts from red green, and blue exposure, respectively. Dye amount is directly proportional to incident light intensity. The negative image is inverted by optically printing onto another negative-acting motion picture film or by processing downstream of a scanner in order to render it suitable for positive display. Photographic color positive films create complementary cyan, magenta, and yellow dye amounts from red, green, and blue exposure also, but the dye amount is inversely proportional to incident intensity. This allows for the positive image to be directly projected in theatrical distribution or to be scanned without a required polarity inversion.
For theatrical exhibition of images captured by film but digitized for the purpose of creative enhancement or the addition of visual effects, calibrated electro-optical scanners are used to convert film density to digital data suitable for electronic display (monitor or digital cinema, for example) or to digital data suitable for driving a film recorder to produce a reproduction of the original image on another piece of film. In the case of the film recorder example, the new film image produced can be used to generate distribution prints for exhibition in typical motion picture cinema theaters.
For broadcast television exhibition, electro-optical scanners known as telecines are used to convert film density to voltage signals suitable for driving a typical display/broadcast monitor.
When images captured with conventional motion picture origination film are chemically developed, and then optically transferred to print film or electro-optically scanned for electronic processing/display, they produce an image appearance that is primarily inherent of the “origination” film's tone, color, sharpness and texture (i.e. graininess). These “origination film attributes” all contribute to what cinematographers describe as the “film look.” A number of origination films are manufactured for the purpose of producing several “film looks” to satisfy the various creative needs of cinematographers. The unique look of each of these films is primarily determined by the type of light sensitive (spectral) and image processing chemistry incorporated into the film's color recording emulsions. If an alternate “image processing” means (method) were available to reproduce the many origination film looks without having the need for incorporating separate chemistry formulations in each film type, it would be advantageous (e.g. cost/workflow efficiency) to theatrical and television film productions. For instance, producers would no longer have a need to carry/track inventory for the several origination film types to satisfy the production needs (looks) of the cinematographer.
Also by removing the requirement for an origination film stock to possess “film-look-ready” images for either optical printing steps or electro-optical scanning steps, the origination film chemistry could now be altered and aimed more efficiently at capturing a higher degree of scene information, as well as producing the corresponding developed image with optical attributes that would allow for greater information to be extracted (being better matched to a scanner's electro-optical characteristics). The film would need to be scanned (a type of “scan-only” film) so that the “image processing” chemistry that was removed (to give an origination film its unique look) could now be applied during a post image processing stage (via software and/or hardware) to achieve the various origination “film stock” looks. The image processing would implement algorithms (in the form of mathematical matrices, 3D look-up tables, or equations) with parameters/values assigned to reproduce the unique film look associated with the particular image processing chemistry formulation that exists in a conventional origination film type. This type of film would allow implementing the same type of image processing algorithms and functional workflow as described in the electronic capture/processing system of U.S. Pat. No. 6,269,217 B1 (after having undergone chemical development and optical scanning). This reference describes not only color, tone, sharpness and texture (graininess) processing, but also the processing needed to compensate for geometric (framing) and psychophysical viewing phenomena associated with image viewing conditions as well as the processing required to prepare images for specific display devices. Relative to current electronically captured images, origination film emulsion technology (e.g. Kodak ECN films) used in motion picture films can reproduce color image data with wider scene exposure latitude (especially in scene highlights and overexposures) and red/green/blue spatial resolution, in addition to being able to provide a longer archival image record (optical) than the current magnetic (e.g. tape) or electro-optic (e.g. optical disk) media used to record/store the image signals from electronic motion cameras.
Several valid examples of related prior art have been investigated to determine their relevancy to the present invention. Digital image processing may take place on video images, as described in U.S. Pat. Nos. 5,335,013; 5,475,425 and 5,831,673, in order to emulate the broadcast look of film or the look of film after it has been through a telecine transfer. These patents describe systems for rendering the output of a video camera to simulate the visual appearance of motion picture film that has been transferred or converted to a video signal to be output directly for television broadcasting or recording on video tape. Further, the above-cited prior art teaches three components for the emulation of the look of broadcast motion picture film. One component deals with the conversion of the video or digital material into various video formats from either 30 frames per second (fps) or 24 fps origination rate. The second component allows for the selective addition of filtered random-type noise to the electronically captured images to give the appearance of motion picture film grain. The third component allows for the alteration of the apparent contrast of the video image so the desired broadcast film appearance may be obtained. More specifically, in the '013 patent a gray scale modifier is used as a look-up table (LUT) and the operator can choose between a variety of curves (% light level vs. video level) stored in programmable read-only memory (PROM) to reflect different film types or achieve different photographic effects. The desired curve is selected by pressing a switch on the hardware. None of these patents refers to a system for manipulating images to match the color/tone characteristics of photographic motion picture films where the starting image is derived from an electro-optical scan of a scan-only film. Further, none reference the electronic processing of scanned images where the primary image space is data and not video.
U.S. Pat. Nos. 5,140,414; 5,374,954; and 5,406,326 (each issued to Mowry) represent a family of related post-production video technology that seeks to arrive at an aesthetically acceptable simulation of the appearance that images originated on different motion picture film stocks would embody after telecine “flying spot scanner” transfer to video from taped high definition video originated images. U.S. Pat. Nos. 5,457,491 and 5,687,011 further extend the concept to providing emulations of images captured on one medium from the capture of the image on a second medium, presumably to include any film capture medium. One component of this prior art technology deals with the conversion of the video-originated material through a LUT that is based on color temperature of the scene lighting, scene brightness and selected f-stop setting. The conversion values in the LUT are derived by filming color charts and gray-scale charts, obtaining a digital representation of the film component responses of the charts from telecine transfer of the film to videotape, and then charting the telecine-derived component responses against video originated images of the same charts under identical lighting conditions. Another component of this prior art technology allows for optically superimposing selected film grain patterns to the video images. The final simulated video image is either recorded as a high definition signal, or converted to an NTSC signal and broadcast or displayed.
In the latter two of the aforementioned Mowry patents, the digitized video signal may be sent to a film recorder, which reproduces the component-modified images onto a selected, reversal film stock. The film is chemically processed with a film processor and then optically projected, or scanned to video, digital video, or other electronic media. However, if the film recording option is employed, these patents specify that it is important that the telecine-derived LUT used in the component modification involves response data which compensates for the inherent color response of the film stock on which the images are being digitally recorded.
In all of the Mowry patents, the creation of image processing LUTs capable of transforming image data representative of the characteristics of one medium to characteristics representative of a second medium, those characteristics including color and tone among others, requires that reference images be necessarily captured on both media to provide statistical basis for the relationship algorithm.
U.S. Pat. Nos. 6,771,323, and 5,840,470 teach how fundamental image characteristic data can be used to provide device-independent or device-dependent intermediate data appropriate for simulating the image characteristics of multiple capture or display media and devices, including film, computer monitors, or video signals from a given image capture device or media. It focuses on capturing imaging device properties such as white level, black level, color level, linearity, and frequency response, and adding those image characteristics to primary scene content to yield enhanced content. It does not, however, reference the full gamut of motion picture color film imaging characteristics necessary to provide completely independent intermediate data, specifically those relevant to the properties of color reproduction. Further it does not acknowledge how some intermediate image spaces, such as scene exposure or scene luminance, must still be qualified by the spectral response properties of imaging devices and converted appropriately to provide full emulation of alternate imaging devices.
Another prior art example of post-production digital image processing, where the digital image could originate from an electronic camera or an electro-optical scan of a film frame, exists in certain current image manipulation software packages. FIG. 1 shows a schematic diagram of one such example of this type of prior art processing, namely, histogram equalization. The histogram equalization method requires, for every frame 1 of a digital image that is to be manipulated, a scanned frame 2 of a reference film preferably with, for optimal results, the same scene content. Some resizing constraints 3 might also have to be met, depending on the software, because the digital image and the scanned film image will most likely not be the same size. Then, with these two input images, a well-known cumulative histogram equalization process 4 is performed to manipulate the digital images closer to a film tonescale and color, thereby providing output manipulated images 5. This method, however, is not optimal for digital images because its inputs can be of mixed formats, especially in the case electronically captured images: some form of RGB exposures from the electronically captured images versus the scanned film densities. As a result, it is impossible to optimally alter the tonescale and color of the electronically captured image to emulate scene exposure as seen by film. For digital images created from electro-optical scans of film frames, the technique also presents problems as only scene color and tone content represented in the references frames can be properly compared. This necessarily excludes specific scene tone and color gamut information not represented in the reference frames. A final limitation of the method is that a reference frame is always required to produce the desired result, leading to impractical image capture scenarios in most common applications.
In a combined approach, U.S. Pat. No. 5,319,465 describes a method using modified camera production and modified post-production processes and equipment to create filmic images. Specifically, the method includes the steps of shooting a benchmark comprising a gray scale chart, a color test chart and two backfocus charts with both a film and a video camera with comparable scene lighting and depth of field. Once the film test benchmark is shot, the film is transferred to videotape utilizing a telecine apparatus, with settings indicative of an industry standard set-up film. The video camera image's hue, saturation, luminance and gamma levels are manipulated to color correct the video camera image to look visually like the transferred film image. After the videotape has been edited, the videotape undergoes color correction in which the videotape benchmark is corrected to match the film test benchmark.
The prior art is generally trying to emulate the look of film after it has been telecine-transferred to video. This is desirable to some extent because the telecine system does have some film attributes when broadcasted. However, the prior art neglects the emulation of the look of film origination, as if a negative film has been directly printed and projected through a motion picture system. This is particularly desirable where the digital output is recorded on film for projection. Where the prior art does deal with film recording, as in the latter two of the above-mentioned Mowry patents, it does so in the context of a telecine-transferred benchmark. Moreover, when the prior art alters the tone scale and color of a video or digitally captured image to emulate a film, it is done on the telecine-transferred benchmark. This is an imperfect alteration because it cannot operate upon the scene exposure as seen by a film.
What is needed is a system that correctly emulates the look of photographic film origination, when the origination film is now a “scan-only” type of film that lacks the image processing chemistry to provide the image look associated with a specified photographic origination film stock, particularly as to the emulation of film tonescale and color reproduction, as if a photographic origination film has been directly printed and projected through a motion picture system.